Generally, an audio signal includes a downmix signal and an ancillary data signal. And, the ancillary data signal can include a spatial information signal and an extension signal. In this case, the extension signal means an additional signal necessary to enable a signal to be reconstructed close to an original signal in generating a multi-channel signal by upmixing the downmix signal. For instance, the extension signal can include a residual signal. The residual signal means a signal corresponding to a difference between an original signal and a coded signal. In multi-channel audio coding, the residual signal is usable for the following cases. For instance, the residual signal is usable for compensation of an artistic downmix signal or specific channel compensation in decoding. And, the residual signal is usable for both of the compensations as well. So, it is able to reconstruct an inputted audio signal into a signal closer to an original signal using the residual signal to enhance sound quality.